IDENTIFICATION  OF  MEASURES  RELATED  TO 
CROSS-CULTURAL  COMPETENCE 


K.G.  Ross 
C.A.  Thomson 

Institute  of  Simulation  and  Training 
University  of  Central  Florida 


DEFENSE  EQUAL  OPPORTUNITY  MANAGEMENT  INSTITUTE 
DIRECTORATE  OE  RESEARCH 


Submitted  to  Dr.  Daniel  P.  MeDonald,  Direetor  of  Researeh 

Spring  2008 


DEOMI  Internal  Report  CCC-08-2 


Report  Documentation  Page 

Form  Approved 

0MB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  0MB  control  number. 

1.  REPORT  DATE 

2QQg  2.  REPORT  TYPE 

3.  DATES  COVERED 

00-00-2008  to  00-00-2008 

4.  TITLE  AND  SUBTITLE 

Identification  of  Measures  Related  to  Cross-Cultural  Competence 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

University  of  Central  Florida, Institute  of  Simulation  and  Training, 4000 
Central  Florida  Blvd,Orlando,FL,32816 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF:  17.  LIMITATION  OF 

_ _ _  ABSTRACT 

18.  NUMBER  19a.  NAME  OF 

OF  PAGES  RESPONSIBLE  PERSON 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE  Same  aS 

unclassified  unclassified  unclassified  Report  (SAR) 

25 

Standard  Form  298  (Rev.  8-98} 

Prescribed  by  ANSI  Std  Z39-18 


Cognitive  Performance  Group 


Review  of  Cross  Cultural  Competence  Measures 


Introduction 

Purpose 

This  task,  the  first  of  five  tasks  in  a  project  to  support  Cultural  Readiness  for  the 
Department  of  Defense,  represents  the  first  step  in  the  development  of  a  “paper  and 
pencil”  questionnaire  measure  of  cultural  competency.  The  purpose  of  this  task  was  to 
identify  measures  that  are  reliable  and  valid  for  the  measurement  of  cultural  competency. 
Thus,  our  approach  was  to  identify  existing  measures  in  published  peer-reviewed 
literature  in  order  to  understand  the  available  instruments,  the  dimensions  and  constructs 
they  measure,  the  purposes  and  previous  uses  of  these  metrics,  and  their  reported 
psychometric  properties.  This  document  provides  an  overview  of  the  project  as  a  context 
for  this  task,  provides  the  findings  of  this  task,  and  discusses  next  steps  in  the  project. 

Project  Overview 

This  initial  project  supports  the  assessment  of  capabilities  and  requirements  leading  to  the 
development  of  a  common  framework  in  order  to  “set  the  stage”  for  input  to  policy,  as 
well  as  for  research  and  training  efforts  by  the  emerging  Department  of  Defense  Cultural 
Center  of  Excellence.  Thus,  the  overall  purpose  of  this  project  is  to  develop  a  framework 
of  cultural  competence  and  its  related  measures. 

The  project  includes  five  tasks: 

1)  Identify  Measures  Related  to  Culture 

2)  Establish  Key  Operational  Definitions 

3)  Review  of  the  Eiterature 

4)  Collection  of  Baseline  Measures 

5)  Preliminary  Report  of  Results 

Identification  of  existing  measures  is  the  first  step  in  the  development  of  a  measurement 
tool  and  in  the  development  of  a  framework  that  includes  and  links  the  constructs  that  are 
part  of  cultural  competency.  To  clarify  how  this  first  step  supports  the  development  of  a 
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measurement  tool  and  how  we  conceive  of  this  task  in  the  context  of  the  overall  project, 
we  briefly  review  the  process  for  metric  development  in  the  next  section. 

Methodology  for  Developing  a  Measure  of 
Cross  Cultural  Competence 

To  develop  a  questionnaire  that  is  psychometrically  valid  and  reliable  -  that  measures 
what  it  purports  to  measure  with  a  high  degree  of  reliability  and  consistency  -  researchers 
must  adhere  to  the  five  steps  of  questionnaire  design.  These  include  conceptualization, 
prototype  construction,  questionnaire  tryout,  item  analysis,  and  revision  (Cohen  & 
Swerdlik,  2002).  The  conceptualization  phase  often  starts  with  a  review  of  the  literature 
of  the  existing  questionnaires  that  have  been  designed  to  measure  the  construct  of 
interest,  in  this  case,  cultural  competency  and  its  related  manifestations.  This  task 
addresses  this  first  step  of  metric  design.  Conducting  such  a  review  might  reveal  that 
such  measures  leave  something  to  be  desired  in  terms  of  psychometric  soundness,  which 
serves  as  the  stimulus  for  developing  the  new  measure  (p.  189,  Cohen  &  Swerdlik,  2002). 
In  addition,  we  believe  that  the  existing  measures  may  leave  something  to  be  desired  in 
terms  of  the  constructs  assessed  as  well  as  the  relevance  of  existing  measures  for  the 
purposes  of  the  current  assessment  requirements. 

In  addition  to  literature  review,  in  order  to  conceptualize  the  overall  construct  of  interest 
(i.e.,  cultural  competency),  some  basic  issues  must  be  addressed.  These  include  coming 
up  with  a  precise  definition  of  what  exactly  is  being  measured,  how  it  will  be  measured, 
the  format  of  the  questionnaire,  benefits  to  such  measurement,  and  the  meanings  of  the 
scores  derived,  among  others.  First  and  foremost  in  this  process,  the  conceptual  criteria 
should  be  carefully  identified  to  include  all  of  the  important  dimensions  of  performance. 
This  will  involve  not  only  theoretically-based  hypotheses  of  important  constructs  that 
make  up  a  measure  of  cultural  competency,  but  also  detailed  explications  of  what 
performance  success  looks  like  that  can  be  tied  to  all  of  the  important  and  relevant 
outcomes  of  mission  effectiveness.  As  part  of  conceptualizing  what  exactly  is  being 
measured,  for  Task  2  we  plan  to  go  beyond  the  theoretical  constructs  that  can  be 
demonstrated  psychometrically  in  order  to  create  those  detailed  explications  of  mission 
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relevant  performance  through  the  conduct  of  open-ended  interviews  focused  on  critical 
incidents  ( Cl)  related  to  cultural  competency.  Interviews  can  be  conducted  with  people 
who  are  identified  through  existing  questionnaires  or,  as  we  plan  to  do  in  this  case,  with 
people  who  are  believed  to  possess  varying  degrees  of  the  competency  in  question  as 
gained  by  field  experience.  These  interviews  are  expected  to  lead  to  more  targeted 
structured  interviews  which  will  more  directly  assess  the  construct  of  interest,  leading  to 
additional  items  for  the  measurement  instrument  as  well  as  “thick”  descriptions  of 
challenges  and  competencies  in  the  field. 

Thus,  we  are  undertaking  a  two-tiered  approach  to  the  conceptualization  of  operational 
definitions  of  cultural  competency.  Cultural  competency  must  be  defined  for  the  purposes 
of  this  project  both  in  terms  of  the  psychological  variables  that  can  be  measured  in 
questionnaire  formats  and  in  relation  to  the  unique  performance  challenges  required  in 
the  context  of  mission  performance.  This  combination  of  conceptual  criteria,  or 
constructs,  along  with  skill  descriptions  based  on  the  challenges  faced  by  experienced 
members  of  the  military  as  they  make  judgments  and  decisions  and  as  they  communicate 
to  achieve  the  goals  of  current  missions  will  form  our  operational  definitions  of  cultural 
competency  in  Task  2. 

The  ordering  of  these  first  two  steps  (i.e.,  development  of  conceptual  criteria  and 
exploration  of  field  performance  effectiveness)  is  critical  in  order  to  prevent  the  all-too- 
common  practice  of  using  outcome  criterion  measures  simply  because  they  are  available 
or  easily  developed.  Beginning  with  a  hasty  “front-end  analysis”  can  lead  to  erroneous 
and  misleading  training  objectives  and  policies,  as  well  as  a  shaky  foundation  for  future 
research.  As  noted,  we  believe  that  there  may  be  two  important  and  complementary 
aspects  to  the  assessment  of  competence  that  we  need  to  pursue.  Interviews  during  Task  2 
will  inform  our  understanding  of  this  issue.  The  results  of  our  questionnaire  development 
efforts  combined  with  the  results  of  our  Cl  interviews  will  allow  us  to  determine  the 
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value  of  each  approach,  construct  an  initial  questionnaire,  and  construct  an  initial 
approach  to  performance-based  assessment  as  well/ 

The  next  step  in  questionnaire  development  is  the  creation  of  a  prototype  for 
administration  in  a  pilot  study.  Because  cultural  competency  is  multidimensional,  many 
more  questions  than  needed  should  initially  be  included  in  such  as  prototype.  This  will 
allow  us,  during  pilot-testing,  to  assess  as  many  possible  manifestations  of  different 
content  areas  that  may  be  relevant  to  the  construct  of  interest.  Therefore,  as  large  an  item 
pool  of  questions  as  possible  will  be  used  to  ensure  adequate  content  coverage  (as  well  as 
construct  validity).  This  process  “may  entail  the  creation,  revision,  and  deletion”  of  many 
items,  “as  well  as  literature  reviews,  experimentation,  even  soul-searching”  (p.  191, 
Cohen  &  Swerdlik,  2002).  This  approach  of  creating  a  large  pool  of  questions  will  be 
tempered  by  the  knowledge  that  there  are  practical  limits  to  people  completing 
questionnaires  both  from  questionnaire  development  and  implementation  points  of  view. 

In  terms  of  the  construction  of  our  pilot  instrument,  we  will  adhere  to  the  generally 
accepted  concepts  of  questionnaire  construction.  Scaling  is  an  important  part  of 
construction  involving  setting  rules  for  assigning  numbers  in  measurement.  A  five-  to 
seven-point  Likert  scale  is  generally  used,  as  this  consistency  makes  it  easier  for 
respondents  to  complete  the  questionnaire,  makes  it  easier  to  score,  and  enhances 
validity.  Additionally,  items  must  be  phrased  carefully,  simply,  and  unambiguously,  as 
recommended  by  psychometricians  (Rust  &  Golombok,  1989).  As  such,  the  use  of 
double-negatives  must  be  avoided  and  each  item  must  only  ask  about  one  issue.  Another 
caution  to  keep  in  mind  is  that  an  understanding  of  any  key  concepts  should  not  be 
assumed  in  participants;  therefore,  questions  should  not  ask  respondents  directly  about 
the  constructs  of  interest,  which  may  contain  unfamiliar  terms  to  them.  Social  desirability 
bias  is  usually  managed  by  instructions  to  avoid  spending  too  long  on  any  one  question 
and  by  emphasizing  that  the  first  response  is  usually  the  best  response. 


*  Initial  interviews  that  can  be  funded  as  part  of  this  project  will  be  limited  and  will  allow  us  to  understand 
the  nature  of  the  challenges  and  competencies,  but  will  not  provide  results  that  are  extensive  enough  to 
fully  develop  an  approach  to  performance-based  assessment  or  a  complete  model  of  performance-based 
competency. 
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Questionnaire  tryout  is  the  third  phase  in  developing  a  psychometrically  sound 
questionnaire.  Having  created  a  large  pool  of  items,  it  is  necessary  to  administer  the 
questionnaire  to  as  large  of  a  pool  of  participants  as  possible,  certainly  no  fewer  than  five 
participants  per  item  (preferably  ten  per  item),  should  be  available.  With  any  smaller 
sample  size,  we  will  run  the  risk  of  “phantom  factors,”  nonexistent  factors  that  emerge  in 
factor  analysis  when  a  sample  size  is  too  small.  Given  the  access  to  large  numbers  of 
potential  participants  through  the  existing  survey  population,  we  do  not  anticipate  a 
problem  with  sample  size.  However,  given  the  restrictions  on  how  many  items  can  be 
added  to  the  existing  survey  administrations,  there  may  be  difficulty  in  piloting  an 
adequate  item  pool  during  development. 

Item  analysis,  which  involves  complex  statistical  procedures,  follows.  Basically,  this 
involves  statistics  that  yield  item  differentiation,  item-validity,  and  item-reliability 
indices  to  determine:  (1)  if  items  differentiate  or  discriminate  well  between  those  who  are 
high  versus  low  on  the  particular  characteristic  being  measured  (i.e.,  cultural 
competence);  (2)  the  degree  to  which  each  item  measures  what  it  purports  to  measure; 
and  (3)  the  internal  consistency  of  the  questionnaire  as  a  whole.  Further,  in  order  to 
assess  the  inter-item  consistency,  whether  items  appear  to  be  measuring  the  same  thing  as 
a  whole,  factor  analysis  is  used.  Subjecting  findings  to  an  exploratory  and  confirmatory 
factor  analysis  will  serve  to  refine  our  operational  definitions  and  provide  empirically- 
based  definitions  of  the  constructs. 

Having  conceptualized  the  new  questionnaire,  constructed  it,  tried  it  out,  and  analyzed 
the  items,  the  next  step  is  to  act  upon  all  of  the  information  obtained  in  the  analyses  and 
mold  the  questionnaire  into  its  final  form.  Many  of  the  items  will  have  been  eliminated 
and  others  rewritten  based  upon  such  analyses  as  to  which  items  were  the  weakest.  This 
is  the  advantage  of  a  large  item  pool  -  many  poorer  items  can  be  eliminated,  making  the 
final  form  as  robust  as  possible. 
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Existing  Measures  Reiated  to  Cuiturai  Competence 

Our  review  describes  13  studies  covering  a  range  of  constructs  related  to  cultural 
competency,  and  the  metrics  that  were  used  for  a  variety  of  purposes.  Only  those  studies 
that  reported  psychometric  data  (i.e.,  reliability,  validity)  were  included  in  our  review. 
This  necessarily  limited  the  breadth  of  studies  included  herein.  As  can  be  seen  from  the 
Appendix,  there  are  several  measures  related  to  cultural  competency  that  have  been 
developed  over  the  years.  The  measures  developed,  and  the  constructs  they  assess, 
depend  upon  the  purpose  and  type  of  competency  required  as  well  as  the  population  of 
interest. 

Much  of  this  research  is  business  driven.  For  instance,  the  explosive  growth  in 
globalization  has  led  to  a  growing  number  of  people  with  international  assignments, 
international  joint  ventures  and  people  moving  to  other  countries  to  work.  Because  of  this 
phenomenon,  cultural  competence  has  become  increasingly  important  in  business 
management.  Thus,  several  measures  have  been  developed  for  business  purposes  (e.g., 
Koester  &  Olebe,  1988;  Matsumoto  et  ah,  2001;  Van  der  Zee  &  Van  Oudenhoven,  2000). 

Because  administrators  of  mental  health  systems  and  agencies  wish  to  hire  culturally 
competent  providers  and  to  train  their  providers  to  be  culturally  competent  (Sue,  2003), 
several  measures  were  developed  and  validated  for  measuring  multicultural  competence 
(a  similar,  related  construct)  as  it  relates  to  psychologists,  therapists,  social  workers,  and 
mental  health  counselors  who  treat  diverse  populations  of  patients.  Most  of  these  are  self- 
report  (e.g.,  D’Andrea,  Daniels,  &  Heck,  1991;  Holcomb-McCoy  &  Myers,  1999; 
LaFromboise  et  ah,  1991;  Ponterotto  et  ah,  1996;  Sodowsky,  Taffe,  Gutkin,  &  Wise, 
1994);  however,  we  found  one  measure  that  is  a  “consumer”-based  measure  (e.g., 
Cornelius,  Booker,  Arthur,  Reeves  &  Morgan,  2004). 

Beyond  the  divergent  purposes  of  the  particular  instruments  developed,  the  measures  also 
differ  in  terms  of  the  constructs  they  assess.  That  is,  some  were  designed  to  measure 
hypothesized  predictors,  or  antecedents,  of  behavior,  such  as  personality  traits.  These 
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traits  are  expected  to  lead  individuals  to  perform  certain  behaviors;  these  behaviors  are 
then,  in  turn,  expected  to  lead  to  the  desired  outcomes  that  define  effectiveness  (for  the 
particular  purpose  they  were  designed).  However,  each  link  in  this  causal  chain  is 
oftentimes  not  tested  empirically  and/or  statistically  by  researchers.  Such  predictors  of 
performance  include  those  that  are  cognitive-based  (e.g.,  knowledge)  as  well  as 
personality-based  (e.g.,  openness,  extraversion,  etc.).  Therefore,  it  is  not  clear  how  many 
items,  if  any,  of  the  existing  measures  we  can  use  in  our  prototype  and  how  many  items 
we  will  have  to  develop  ourselves.  It  is  simply  too  early  to  tell. 

Conceptually,  the  predictors  of  the  processes  of  performance  lead  to  the  outcomes  that 
enable  mission  success.  However,  we  must  work  backward,  starting  with  the  conceptual 
criteria  that  define  mission  success,  in  order  to  ensure  that  our  measures  are  relevant, 
neither  deficient  nor  contaminated,  at  each  step.  Often,  researchers  do  not  do  this  and 
skip  steps,  coming  up  with  metrics  that  predict  processes  and  measure  the  antecedents  of 
performance,  but  if  these  processes  do  not  lead  to  important  outcomes  (i.e..,  results), 
there  is  no  practical  value  in  them.  Only  after  identifying  desired  mission-related 
performance  outcomes  and  those  constructs  that  are  conceptually  related  to  those 
outcomes  can  we  begin  to  develop  an  instrument  to  specifically  identify  individuals  who 
possess  the  relevant  characteristics  (antecedents  of  performance)  that  actually  lead  to  the 
results  we  desire. 

Next  Steps  in  the  Development  of  a  Cross  Cultural  Competence 
Framework  and  Related  Measures 

Our  next  step  in  the  project  is  Task  2,  the  establishment  of  operational  definitions.  This 
task  is  essentially  the  establishment,  a  priori,  of  the  constructs  we  believe  we  will  be 
measuring  based  on  the  literature  and  further  explication  of  cultural  competency  through 
Cl  interviews.  As  stated  above,  we  expect  that  we  will  find,  through  analysis  of  the 
measures  identified  here  and  the  results  of  the  interviews,  that  there  are  constructs  that 
should  be  measured  via  questionnaire  administration  and  others  that  are  skill-based  and 
must  be  measured  in  performance  environments  in  order  to  ground  competency  within 
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mission  effectiveness  and  readiness.  Therefore,  the  results  of  Task  2  will  provide  us  with 
the  direction  needed  to  address  Task  3,  the  literature  review. 

In  Task  3,  we  will  further  finalize  the  constructs  we  wish  to  measure,  as  to  their 
theoretical  underpinnings,  and  format  our  prototype  questionnaire  for  pilot  administration 
in  Task  4.  We  will  describe  our  rationale  for  the  resulting  pilot  questionnaire,  and  discuss 
the  research  and  practical  issues  related  to  performance-based  measurement. 
Additionally,  Task  4  requires  the  collection  of  “baseline  measures.”  We  understand 
baseline  measures  to  mean  establishment  of  some  preliminary  normative  data  in  the 
existing  population.  To  accomplish  this  task,  we  must  conduct  several  administrations  to 
complete  the  questionnaire  development  and  then  collect  initial  baseline  data  using  the 
final  form. 

This  project  concludes  with  our  final  report  on  the  results  of  this  initial  effort,  the 
findings  from  each  task,  and  the  overall  results  and  conclusions  with  our 
recommendations  for  a  framework  of  cultural  competence  and  its  measurement.  We  will 
also  include  results  generated  from  the  administration  of  the  questionnaire’s  final  form, 
as  well  as  recommendations  for  further  development  on  the  future  utilization  and 
application  of  findings  from  this  project. 
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Appendix:  Table  of  Measures  Related  to  Cross  Cultural  Competence 


Title  of  Study 

Measure(s)  used 

Dimensions 

Assessed 

Psychometric 

Properties/Results 

Authors  & 
Year 

Comments 

Multicultural 

Competency 

Instrumentation: 

A  Review  and 
Analysis  of 
Reliability 
Generalization 

•  Multicultural  Counseling 
Inventory  (MCI; 
Sodowsky,  Taffe, 

Gutkin,  &  Wise,  1994) 

•  Multicultural  Awareness 
Knowledge  and  Skills 
Survey  (MAKSS; 

D ’Andrea,  Daniels,  & 
Heck,  1991) 

•  Cross-Cultural 

Counseling  Competence 
Inventory-Revised 
(CCCI-R;  LaFromboise 
et  al.,  1991) 

•  Multicultural  Counseling 
Knowledge  and 

Awareness  Scale 
(MCAS:  B;  Ponterotto 
et  al.,  1996) 

•  Multicultural  Counseling 
Competence  and 

Training  Survey 
(MCCTS;  Holcomb- 
McCoy  &  Myers,  1999). 

•  Multicultural 

Competency  (in  the 
context  of 
counseling) 

•  Acceptable  internal 
consistency 
coefficients  over  time 
and  across 
populations. 

•  Specifically,  the  MCI, 
MAKSS  (currently  the 
MAKSS-CE-R), 
CCCI-R,  and  MCKAS 
may  appropriately  be 
used  across  many 
populations  and 
settings. 

Dunn,  Smith, 

&  Montoya 
(2006) 

•  This  is  a  comprehensive 
review  of  internal 
consistency  reliability  of 
multicultural 
competency 
instrumentation  out  of 
the  more  than  800 
manuscripts  that  have 
addressed  multicultural 
competence  in  mental 
health  professional 
training  and  assessment. 

14 


Cognitive  Performance  Group 


Review  of  Cross  Cultural  Competence  Measures 


Title  of  Study 

Measure(s)  used 

Dimensions 

Assessed 

Psychometric 

Properties/Results 

Authors  & 
Year 

Comments 

Appreciating 
Similarities  and 
Valuing 

Differences:  The 
Miville -Guzman 
Universality- 
Diversity  Scale 

•  Miville -Guzman 

Universality-Diversity 
Scale  (M-GUDS),  a  45- 
item  scale  administered 
to  4  samples. 

•  Universality- 
Diversity 

Orientation  (UDO), 
a  construct  that  was 
defined  in  this 
study  as  an  attitude 
of  awareness  and 
acceptance  of  both 
the  similarities  and 
differences  that 
exist  among 
people. 

•  Significant  correl¬ 
ations  in  theoretically 
predicted  ways  with 
measures  of  racial 
identity,  empathy, 
healthy  narcissism, 
feminism,  androgyny, 
homophobia,  and 
dogmatism. 

•  Discriminant  validity 
displayed  by  scale 
failing  to  correlate 
with  Scholastic 
Achievement  Test 
Verbal  scores, 
although  mixed  results 
were  obtained  with 
social  desirability. 

•  Internal  consistency 
and  retest  reliability 
ranged  from  .89  to  .95 

•  In  sum,  the  data 
suggest  considerable 
reliability  and  initial 
construct  validity  for 
the  M-GUDS. 

Miville, 
Holloway, 
Gelso,  Pannu, 
Liu,  Touradji, 

&  Fuertes 
(1999) 

•  Vontress  (1996) 
proposed  that  people  are 
the  products  of  several 
cultures  that  interact 
with  each  other:  (a) 
universal,  (b)  ecological, 
(c)  national,  (d)  regional, 
and  (e)  racioethnic. 

•  Universal  culture  refers 
to  "the  all  encompassing 
humanness  in  each  of  us 
which  pervades  all 
cultures. 

•  No  matter  what  the 
conditions  are  under 
which  people  live,  they 
must  adjust  to  the  fact 
that  they  are  human 
beings"  (Vontress,  1996, 
p.  164). 
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Title  of  Study 

Measure(s)  used 

Dimensions 

Assessed 

Psychometric 

Properties/Results 

Authors  & 
Year 

Comments 

Development  of 

Counseling 

Trainees' 

Multicultural 

Awareness 

Through 

Mentoring 

English  as  a 

Second  Language 
Students 

•  Multicultural  Counseling 
Inventory  (MCI; 

Sodowsky  et  ah,  1994) 

•  Multicultural  Social 
Desirability  Scale  (MCD; 
Sodowsky,  Kuo-Jackson, 
Richardson,  &  Corey, 

1998) 

•  White  Racial  Identity 
Attitudes  Scale  (WRIAS; 
Helms,  1990) 

•  Multicultural 
Competency- 
multicultural  skills, 
awareness, 
knowledge,  and 
relationship. 

•  Multicultural 
Desirability  -  the 
expression  of 
thoughts  and 
behaviors 
motivated  by  a 
desire  to  avoid  the 
appearance  of 
stereotype  bias. 

•  Racial  Attitudes  — 
positive  attitudes  of 
Whites  towards 
African  Americans 

•  MCI  -  Cronbach's  alpha 
coefficients  have  been 
stable  across  studies, 
with  the  average  alpha 
of  .87  for  the  full  scale, 
.75  for  skills,  .77  for 
awareness,  .66  for 
relationship,  and  .75  for 
knowledge  (e.g., 
Constantine,  2001; 
Worthington  et  ah, 

2000). 

•  MCD  -  In  previous 
studies,  internal 
consistency  ranged 
from  .75  (Sodowsky  et 
ah,  1998)  to  .73 
(Hansen  et  ah,  2004). 

Roysircar, 

Card, 

Hubbell,  & 
Ortega  (2005) 

•  The  first  author  has 
written  on  trainees' 
practice  of  the  self¬ 
reflexive  process 
(Roysircar,  2003,  2004) 
with  descriptions  of 
trainees'  reflections  on 
critical  incidents  with 
clients  that  resulted  in 
trainees'  "increased 
understanding  of  self  and 
others,  and  a  greater 
appreciation  and  respect 
for  differences" 

(Roysircar,  2003,  p.  34) 
as  well  as  an  increased 
ability  to  "retell  the 
story,  incorporating  the 
client's  worldview  and 
correcting  one's  assump¬ 
tions,  values,  and  biases" 
(p.  663). 

Development  of 
the  Multicultural 
Counseling 
Inventory:  A  Self- 
Report  Measure 
of  Multicultural 
Competencies 

•  Multicultural  Counseling 
Inventory  (MCI) 

•  Multicultural 
Counseling  Skills 

•  Multicultural 
Awareness 

•  Multicultural 
Counseling 
Relationship 

•  Multicultural 
Counseling 
Knowledge 

•  Sufficient  evidence 
was  found  for  the 
existence  of  4  specific 
factors 

•  Results  show  moderate 
to  moderately  high 
internal  consistency 
reliabilities  and 
moderate  interfactor 
correlations. 

Sodowsky, 
Taffe,  Gutkin, 
&  Wise 
(1994) 

•  Original  development  of 
the  Multicultural 

Counseling  Inventory 
(MCI),  a  self-report 
instrument  that  measures 
multicultural  counseling 
competencies. 
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Title  of  Study 

Measure(s)  used 

Dimensions 

Assessed 

Psychometric 

Properties/Results 

Authors  & 
Year 

Comments 

The  Scale  of 

Ethnocultural 

Empathy: 

Development, 

Validation,  and 

Reliability 

•  Scale  of  Ethnocultural 
Empathy  (SEE) 

•  Ethnocultural 
Empathy  (cultural 
empathy)  - 
empathy  toward 
people  of  racial  and 
ethnic  backgrounds 
different  from 
one’s  own 

•  Intellectual 

Empathy  —  the 
cognitive 
component  of 
above  -  the  ability 
to  perceive  the 
world  as  another 
person  does. 

Eour  Eactors  derived: 

•  Empathic  Eeeling  and 
Expression 

•  Empathic  Perspective 
Taking 

•  Acceptance  of  Cultural 
Differences 

•  Empathic  Awareness. 
High  internal  consis¬ 
tency  and  test— retest 
reliability  estimates  were 
found  across  the  three 
studies  as  well  as 
evidence  of  construct 
validity. 

Wang, 
Davidson, 
Yakushko, 
Savoy,  Tan, 

&  Bleier 
(2003) 

•  Quintana’s  (1994)  theory 
of  social  perspective¬ 
taking  operationalizes 
constructs  as  cognitive 
developmental  abilities. 

•  Such  abilities  contain 
associated  develop-mental 
levels:  “awareness  of 
perspectives,  attitudes, 
and  experiences  shared  by 
ethnic  group,”  and  the 
“enhanced  ability  to  take 
the  perspective  of  other 
ethnic  groups”  (p.  163). 

Measuring 

Intercultural 

Sensitivity:  The 

Intercultural 

Development 

Inventory 

(Continued  .  .  .) 

•  Intercultural  Development 
Inventory  (IDI) 

•  Intercultural 
competence  -  the 
ability  to  think  and 
act  in  inter-culturally 
appropriate  ways. 

Eive  factors  derived: 

•  DD  (Denial/Defense) 

(14  items) 

•  R  (Reversal)  (9  items) 

•  M  (Minimization)  (10 
items) 

•  AA  (Acceptance/ 
Adaptation)  (14  items) 

•  EM  (Encapsulated/ 
Marginality)  (5  items) 

•  Scale  reliabilities 
(coefficient  alpha)  were 
0.70  or  higher: 

Hammer, 
Bennett,  & 
Wiseman 
(2003) 

•  Authors  argue  that  greater 
intercultural  sensitivity  is 
necessary  for  greater 
intercultural  competence. 

•  Based  on  the  theoretical 
framework  for 
conceptualizing 
dimensions  of 
intercultural  competence 
by  Bennett  (1986,  1993), 
in  his  developmental 
model  of  intercultural 
sensitivity  (DMIS). 

•  The  DMIS  consists  of 
three  ethnocentric 
orientations,  where  one’s 

17 


Cognitive  Performance  Group 


Review  of  Cross  Cultural  Competence  Measures 


Title  of  Study 

Measure(s)  used 

Dimensions 

Assessed 

Psychometric 

Properties/Results 

Authors  & 
Year 

Comments 

•  DD  scale  -  alpha=0.85 

•  R  scale  -  alpha=0.80 

•  M  scale  -  alpha=0.85 

•  AA  scale  -  alpha=0.84 

•  EM  scale  -  alpha=0.80 

culture  is  experienced  as 
central  to  reality  (Denial, 
Defense,  Minimization), 
and  three  ethnorelative 
orientations,  where  one’s 
culture  is  experienced  in 
the  context  of  other 
cultures  (Acceptance, 
Adaptation,  Integration). 

The  Validity  and 
Reliability  Testing 
of  a  Consumer- 
Based  Cultural 
Competency 
Inventory 

•  Consumer-Based  Cultural 
Competency  Inventory 

Cultural  Competency  - 
Consumer  assessment 
of  the  cultural 
competency  of  mental 
health  providers. 

Subscales  include: 

•  Language  fluency 

•  Understanding 
indigenous 
practices 

•  Acceptance  of 
cultural  differences 

•  Awareness  of 
patient's  culture 

•  Respectful 
behaviors 

•  Patient-provider 
interactions 

•  Consumer 
involvement 

•  Consumer  outreach 

•  Construct  validity  was 
evaluated  by  examining 
the  correlations  between 
each  of  the  8  subscales 
as  well  as  by  factor 
analysis. 

•  Reliability  was  assessed 
using  Cronbach’s  alpha 
for  all  52  items  (alpha  = 
.91,  eta  square  =  .13). 

•  In  sum,  the  overall  scale 
does  a  good  job  of 
measuring  the  concept 
of  cultural  competency 
(as  defined  in  this 
study). 

Cornelius, 

Booker, 

Arthur, 

Reeves  & 

Morgan 

(2004) 

•  Rather  than  a  cross- 
cultural  measure,  this 
was  developed  for  use 
among  subcultures  (e.g.. 
Native  Americans,  Asian 
Americans)  within  the 
American  culture. 

•  Of  interest  in  that  it 
relies  upon  third-party 
(‘consumer’)  assessment 
rather  than  self  report 
data. 
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Title  of  Study 

Measure(s)  used 

Dimensions 

Assessed 

Psychometric 

Properties/Results 

Authors  & 
Year 

Comments 

The  Munroe 
Multicultural 
Attitude  Scale 
Questionnaire:  A 
New  Instrument 
for  Multicultural 
Studies 

•  Munroe  Multicultural 
Attitude  Scale 
Questionnaire 
(MASQUE) 

Multicultural  Attitude 
Transformation: 

•  Knowledge 
(‘know’) 

•  Empathy  (‘care’) 

•  Active  Experience 
(‘act’) 

•  Reliability  of  the  total 
MASQUE  scale  scores 
was  adequate  for 
general  research 
purposes 

•  Only  2  of  the  3 
subscales  were 
acceptable  for 
exploratory  purposes 

•  EEA  showed  that  three 
constructs  emerged, 
and  none  were 
complex 

•  The  low  correlation 
obtained  between  the 
MASQUE  scores  and 
the  Marlowe-Crowne 
may  provide  evidence 
of  validity  in  that 
participants  reflected 
multicultural  attitudes 
rather  than  socially 
desirable  responses. 

Munroe  & 
Pearson 
(2006) 

•  The  purpose  of  this  study 
was  to  develop  and 
initially  validate  scores 
from  an  instrument  that 
used  Banks’  transform¬ 
ative  approach  as  the 
foundation  for  the 
corresponding  stages  of 
development  that  are 
associated  with  attitude 
formation. 
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Assessed 

Psychometric 

Properties/Results 

Authors  & 
Year 

Comments 

Development  and 
Validation  of  a 
Measure  of 
Intercultural 
Adjustment 
Potential  in 
Japanese 
Sojourners:  The 
Intercultural 
Adjustment 
Potential  Scale 
(ICAPS) 

•  Intercultural  Adjustment 
Potential  Scale  (ICAPS) 

•  Examined  item 
content  from  a 
number  of  valid 
and  reliable 
personality 
inventories 
assessing 
psychological 
constructs  related 

to: 

•  Emotion  regulation 

•  Critical  thinking 

•  Openness  and 
Flexibility 

•  Interpersonal 
security 

•  Emotional 
commitment  to 
traditional  ways  of 
thinking 

•  Tolerance  of 
ambiguity 

•  Empathy. 

•  Eight  studies  provide 
evidence  for  internal, 
temporal,  and  parallel 
forms  reliability, 
predictive  validity  with 
both  subjective  indices 
of  adjustment  and 
psychometrically 
standardized  measures, 
peer  ratings,  and  expert 
ratings. 

•  Convergent  validity 
with  a  similar  measure, 
construct  validity  with 
various  personality 
scales,  and  incremental 
validity  are  good. 

•  External  validity  in 
predicting  changes  as  a 
result  of  intercultural 
seminars  and  in 
identifying  experts  who 
work  in  the  intercultural 
field  was  also  provided. 

Matsumoto  et 
al.  (2001) 

•  The  underlying  factor 
structure  suggests  a 
model  of  the 
psychological 
components  related  to 
intercultural  adjustment, 
including  emotion 
regulation,  openness, 
flexibility  and  creativity 
or  personal  autonomy. 
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The  Multicultural 

Personality 

Questionnaire:  A 

Multidimensional 

Instrument  of 

Multicultural 

Effectiveness 

•  Multicultural  Personality 
Questionnaire 

•  Cultural  Empathy 

•  Open-mindedness 

•  Emotional  Stability 

•  Orientation  to 

Action 

•  Adventurousness/ 
Curiosity 

•  Elexibility 

•  Extraversion 

•  Internal  consistencies 
of  the  scales  were 
high,  except  Open- 
mindedness  and 
Flexibility. 

•  Four  reliable  higher- 
level  dimensions  were 
distinguished: 

Openness,  Emotional 
Stability,  Social 
Initiative,  and 

Elexibility 

•  The  correlations 
between  these 
dimensions  and  related 
instruments  were  in 
the  expected  direction. 

•  Predictive  value  of 
multicultural  activity 
and  its  incremental 
value  above  the  Big 

Five  in  predicting 
international 
orientation  and 
aspiration  of  an 
international  career 
were  shown. 

Van  der  Zee 
&  Van 
Oudenhoven 
(2000) 

•  The  Multicultural 
Personality 

Questionnaire  may  be 
used  as  an  instrument  for 
the  selection  of 
expatriates  or  as  a 
diagnostic  tool  for 
assessing  further  training 
needs. 
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The  Worldview 
Assessment 
Instrument 
(WAI):  The 
development  and 
preliminary 
validation  of  an 
instrument  to 
assess  world  view 
components 
relevant  to 
counseling  and 
psychotherapy 

•  The  Worldview 
Assessment  Instrument 
(WAI) 

•  Beliefs  concerning 
the  mutability  of 
human  nature 

•  Agency 
(voluntarism- 
determinism) 

•  Locus  of 
responsibility 

•  Relation  to 
authority 

•  Relation  to  group 
(individualism- 
collectivism) 

•  Metaphysics 
(ontological 
spiritualism- 
materialism) 

•  Scale-score -level 
factor  analysis 
revealed  three  higher- 
order  factors. 

•  The  results  of  Study  5, 
which  developed  the 
final  WAI  scales, 
showed  that 
intercorrelations  for 
scales  were  low, 
suggesting 
orthogonality 
(independence  of 
dimensions) 

•  The  scales 
demonstrated  adequate 
reliability 

•  Further  validation 
efforts  are  suggested. 

Koltko-Rivera 

(2000) 

•  Five  studies  involving 

709  participants  from  4 
U.S.  states  are  described. 

•  Appendixes  G  and  Ft 
contain  a  ready-to- 
administer  WAI  protocol 
and  scoring  key. 

•  Appendix  I  makes 
further  recommend¬ 
ations  for  using  the  WAI 
in  research,  including 
multicultural 
psychology,  among 
others. 

Personality 
Correlates  of  the 
Four-Factor 

Model  of  Cultural 
Intelligence 

(Continued  .  .  .) 

•  Cultural  Intelligence 
(CQ) 

•  Personal  Characteristics 
Inventory  (PCI:  Mount  & 
Barrick,  1995). 

CQ  —  Four  Factors: 

•  Metacognitive  CQ 

•  Cognitive  CQ 

•  Motivational  CQ 

•  Behavioral  CQ 

Personality: 

•  Conscientiousness 

•  Emotional  Stability 

•  Openness 

•  Conscientious  was 
related  to 
metacognitive  CQ 

•  Agreeableness  and 
emotional  stability 
were  related  to 
behavioral  CQ 

•  Extraversion  was 
linked  to  cognitive 

CQ,  motivational  CQ, 
and  behavioral  CQ 

Ang,  Van 

Dyne  &  Koh 
(2006) 

•  Study  examined 
relationships  between 

Big  Five  personality  and 
the  four-factor  model  of 
cultural  intelligence 
(CQ) — metacognitive 

CQ,  cognitive  CQ, 
motivational  CQ,  and 
behavioral  CQ 

•  The  intriguing  finding  of 
this  study  is  that 
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•  Extraversion 

•  Agreeableness 

•  Openness  was  related 
to  all  four  factors  of 

CQ  (metacognitive 

CQ,  cognitive  CQ, 
motivational  CQ,  and 
behavioral  CQ) 

Openness  was  the  only 

Big  Eive  that  was 
significantly  related  to 
all  four  aspects  of  CQ. 

•  Although  this  study  is 
not  a  measure  of  CC,  per 
se,  it  shows  that 
personality  is  related  to 
CQ,  a  component  of  CC. 

The  Behavioral 
Assessment  Scale 
for  Intercultural 
Communication 
Effectiveness 

(Continued  .  .  .) 

•  Behavioral  Assessment 
Scale  for  Intercultural 
Communication 
(BASIC)  Effectiveness 

Eight  dimensions: 

1 .  Display  of  respect  — 
ability  to  express 
respect  for  another 

2.  Interaction  posture  — 
ability  to  respond  in 
non-judgmental  way 

3.  Orientation  to 
knowledge  -  how 
one  explains  the 
world 

4.  Empathy  —  capacity 
to  ‘put  oneself  in 
another’s  shoes’ 

5.  Task  role  behaviors- 
contribute  to  group 
problem-solving 
activities 

•  Results  of  the 

administration  of  the 
BASIC  measure  to 

263  college  students 
demonstrated  that  the 
new  scale  correlated 
highly  with  another 
measure  of 
communication 
effectiveness 

Koester  & 
Olebe  (1988) 

•  The  eight  dimensions  of 
intercultural 
communication 
effectiveness  assessed  by 
the  BASIC  were 
originally  developed  by 
Ruben  (1976). 
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6.  Relational  role 
behaviors  build  or 
maintain  group 
relationships 

1  .Interaction  behavior 
and  management  — 
skills  in  governing 
interactions  to  meet 
needs  &  desires  of 
group  members 

8.  Tolerance  of 
ambiguity  —  ability  to 
react  to  new  and 
ambiguous  situations 
with  little  visible 
discomfort 
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